“On the One Hand…”—Econometrics for Anesthesiologists
Eric Sun, MD/PhD
Department of Anesthesiology, Perioperative and Pain
Medicine
Department of Health Research and Policy
Center for Health Policy/Center for Primary Care and
Outcomes Research
Stanford University
Disclaimer
• The work I am discussing precedes my appointment to the staff of the Council
of Economic Advisers
• I am speaking in my capacity as an assistant professor at Stanford University
• Nothing in this talk should be construed as representing the views of the
Federal Government
2
Disclosures
• No conflicts of interest to report
• I receive funding from the National Institute on Drug Abuse (K08DA042314)
5
6
7
8
minus
________________________________________________
9
minus
________________________________________________
+
+
Growth in use
(chronic)
10
minus
________________________________________________
+
+
Growth in use
(chronic)
Growth in use
(low)
11
minus
________________________________________________
+
+
Growth in use
(chronic)
Growth in use
(low)
Growth in n
(chronic)
12
minus
________________________________________________
+
+
Growth in use
(chronic)
Growth in use
(low)
Growth in n
(chronic)
Growth in n
(low)
13
0
5000
10000
15000
20000
25000
30000
0
10
20
30
40
50
60
70
80
90
100
2001 2003 2005 2007 2009 2011 2013
Annual Utilization (MEQ)
% of Opioid Users
chronic (% of users) non-chronic (% of users)
non-chronic utilization (MEQ) chronic utilization (MEQ)
• Results
• 47% of growth due to increase in number of chronic users
• 42% of growth due to increase in use among chronic users
14
15
Outline of Talk
• Why do we care about retrospective studies?
• Overview of Econometric Methods
16
Outline of Talk
• Why do we care about retrospective studies?
• Overview of Econometric Methods
17
Limitations of Clinical Trials
• Key Benefit: Internal Validity
• Randomization ensures that (on average), treatment and control populations
are similar
• Key Drawback: External Validity
• Homogenous population
• Do not address “real world” concerns such as costs, health care system
• “Machinery” of trials ensure that population likely different from “real world”
population
• Philipson et al. 2011; Rothwell 2006; Patsopoulos, 2011
Limitations of Clinical Trials
Limitations of Clinical Trial
• Studio Test Audience Differs from General Viewing Public in Two Significant
Ways
• Found out about screening
• Often have industry connections/motivated
• Interested enough to attend
• Attending a screening is more costly than watching TV at home (commuting,
interviews, etc)
• Bottom Line: More likely to “like” TV more than the average viewer
• Note that this difference is not simply differences in observable characteristics
Limitations of Clinical Trials
• Similarly, trial patients likely to be different from “real world” patients
• Key is (a) found out about trial and (b) willing to enroll
• More motivated
• More open to the therapy
• Trial protocols tend to exacerbate these differences
• “run in” periods
• “helpful reminders”
• May affect external validity if compliance/adherence are important
Limitations of Clinical Trials
• Clinical trials measure efficacy
• Policymakers interested in effectiveness
• How does treatment work in the context of costs, adherence, etc.
• Secondary data possible better able to address effectiveness
• Rooted in real world
• Must address bias
Outline of Talk
• Why do we care about retrospective studies?
• Overview of Econometric Methods
• Primer on bias
• How can we deal with bias?
23
Outline of Talk
• Why do we care about retrospective studies?
• Overview of Econometric Methods
• Primer on bias
• How can we deal with bias?
24
Quick Primer on Bias
• Consider the following scenario
• Resident see a low MAP, pushes 100mcg phenylephrine
• Doesn‘t notice that his attending (med student?) also pushed 50mcg SNP
Quick Primer on Bias
Confounding Effect
(50mcg SNP)/Bias
UP
True Effect
(100mcg neo)
DOWN
+
0
=
Observed Effect
Thinking about bias
Confounding Effect
(200mcg neo)
UP
True Effect
(100mcg neo)
UP
+
UP!!!
=
Observed Effect
Quick Primer on Bias
• For bias to be important:
• Confounder is unobserved
• Confounder is correlated with treatment
• Potential effect of confounder helps sign bias
• SNP: downward biased (observed effect less than true effect)
• Neo: upward biased (observed effect more than true effect)
Outline of Talk
• Why do we care about retrospective studies?
• Overview of Econometric Methods
• Primer on bias
• How can we deal with bias?
29
Quick Primer on Bias
• How to Address Bias
• Control for bias (e.g., watch what attending is doing)
• Eliminate bias (e.g., IM succinylcholine)
• Quasi-randomization, natural experiments
Controlling for bias
• Basic idea: identify possible confounders and “net out” their potential effect
• Comes with potential cost: decreased effective sample size
A Primer on Bias – Costs of Dealing with Bias
Suppose I want to see if height affects survival
Two heights: short and tall
Sample size =100,000
50,000 tall
50,000 short
A Primer on Bias – Costs of Dealing with Bias
• So, my study will compare survival among 50,000 tall people to 50,000 short
people
• Good power
• Likely to find a significant effect
A Primer on Bias – Costs of Dealing with Bias
Now, suppose I need to control for race
Assume
2 races: White, Asian
most Whites are tall and most Asians are short
A Primer on Bias – Costs of Dealing with Bias
Now comparing “tall” to “short” is not sufficient
Because I’m also comparing “white” to “Asian”
“Controlling” requires that I compare survival among
tall whites to short whites
tall Asians to short Asians
Take average of above
A Primer on Bias – Costs of Dealing with Bias
Controlling for race effectively makes my sample size smaller
Sample size now driven by “exceptions to the rule”
› Need lots of short whites and tall asians
May no longer have statistical power
Controlling for Bias
“..as we know, there are known knowns; there are things we know we know. We
also know there are known unknowns; that is to say there are some things we
know we do not know. But there are also unknown unknowns—the ones we don’t
know we don’t know”
-Donald Rumsfeld
Controlling for Bias
• Most studies typically control/adjust for observable characteristics
• i.e., age, sex, observable indicators of health
• “known knowns”
• But what about unobservable characteristics?
• “known unknowns”
• “unknown unknowns”
Controlling for Bias
• Richness of MPOG data allow investigators to control for unobservables
• Physician fixed effects, hospital fixed effects, year fixed effects
• Adjust for unobservable characteristics unique to the physician, hospital , year
Physician Fixed Effects
Quasi-Randomization
• We know as physicians—much of what determines whether a patient gets
treatment A or B is random
• Attending (consultant) preference
• Distance to nearest hospital with given preferences/characteristics
• Arbitrary cutoffs
• Quasi-randomization attempts to use this randomness to isolate causal effects
Quasi-Randomization
• Instrumental Variables
• Identify something (instrument) that affects whether you get treatment but
which likely has no effect on outcomes
• Distance to nearest hospital with regional anesthesia capabilities
(Neumann, 2014)
• Number of AAs/CRNAs available to do cases on a given day (Sun, 2018)
• Instrument is used to randomize patients
• Regression Discontinuity
• Exploit arbitrary cutoffs to examine treatment effects
• no nerve blocks if plt<100k
• MPOG contains lots of data one could exploit
• Date/Time of surgery (call)
• Lab values
Instrumental Variables
Instrumental Variables
• Anesthesia care can be provided by either a nurse anesthetist (NA), or
anesthesiologist assistant (AA), or physician (MD)
• Is there a difference in outcomes when a NA or AA is supervised by a MD?
• Important policy implications
• NAs can practice in all 50 states, AA in only 16 states+DC
• Key Issue: Cases performed by NAs may differ from those performed by AAs
• Address this issue by exploiting random daily variation in the AA/NA case mix
• Call, vacation, etc
• Unlikely to be association with patient/case severity as these schedules are
determined long in advance
Key Takeaways
• Retrospective/database studies add tremendous value--if you can deal with
bias
• One way to deal with bias is to adjust for potential confounders
• Richness of MPOG data allow for use of specific fixed effects to adjust for
unobservable confounders
• Another way to deal with bias is to consider “real-life” randomizations
• Again, richness of MPOG data provide ways to exploit this
Questions?
46
